Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. RcppArmadillo integrates this library with the R environment and language and is widely used by (currently) 786 other packages on CRAN. A little while ago, Conrad released version 10.1.0 of Armadillo, a a new major release. As before, given his initial heads-up we ran two full reverse-depends checks, and as a consequence contacted four packages authors (two by email, two via PR) about a miniscule required change (as Armadillo now defaults to C++11, an old existing setting of avoiding C++11 lead to an error). Our thanks to those who promptly update their packages truly appreciated. As it turns out, Conrad also softened the error by the time the release ran around. But despite our best efforts, the release was delayed considerably by CRAN. We had made several Windows test builds but luck had it that on the uploaded package CRAN got itself a (completely spurious segfault which can happen on a busy machine building machine things at once). Sadly it took three or four days for CRAN to reply our email. After which it took another number of days for them to ponder the behaviour of a few new deprecated messaged tickled by at the most ten or so (out of 786) packages. Oh well. So here we are, eleven days after I emailed the rcpp-devel list about the new package being on CRAN but possibly delayed (due to that seg.fault). But during all that time the package was of course available via the Rcpp drat. The changes in this release are summarized below as usual and are mostly upstream along with an improved Travis CI setup due to the aforementioned use of the bspm package for binaries at Travis.

Changes in RcppArmadillo version 0.10.1.0.0 (2020-10-09)

• Upgraded to Armadillo release 10.1.0 (Orchid Ambush)
  • C++11 is now the minimum required C++ standard
  • faster handling of compound expressions by trimatu() and trimatl()
  • faster sparse matrix addition, subtraction and element-wise multiplication
  • expanded sparse submatrix views to handle the non-contiguous form of X.cols(vector_of_column_indices)
  • expanded eigs_sym() and eigs_gen() with optional fine-grained parameters (subspace dimension, number of iterations, eigenvalues closest to specified value)
  • deprecated form of reshape() removed from Cube and SpMat classes
  • ignore and warn on use of the ARMA_DONT_USE_CXX11 macro
• Switch Travis CI testing to focal and BSPM

Courtesy of my CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page. If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Old laptop I ve been meaning to get a new laptop for a while now. My ThinkPad X250 is now 5 years old and even though it s still adequate in many ways, I tend to run out of memory especially when running a few virtual machines. It only has one memory slot, which I maxed out at 16GB shortly after I got it. Memory has been a problem in considering a new machine. Most new laptops have soldered RAM and local configurations tend to ship with 8GB RAM. Getting a new machine with only a slightly better CPU and even just the same amount of RAM as what I have in the X250 seems a bit wasteful. I was eyeing the Lenovo X13 because it s a super portable that can take up to 32GB of RAM, and it ships with an AMD Ryzen 4000 series chip which has great performance. With Lenovo s discount for Debian Developers it became even more attractive. Unfortunately that s in North America only (at least for now) so that didn t work out this time.

Enter Tongfang I ve been reading a bunch of positive reviews about the Tuxedo Pulse 14 and KDE Slimbook 14. Both look like great AMD laptops, supports up to 64GB of RAM and clearly runs Linux well. I also noticed that they look quite similar, and after some quick searches it turns out that these are made by Tongfang and that its model number is PF4NU1F. I also learned that a local retailer (Wootware) sells them as the Wootbook. I ve seen one of these before although it was an Intel-based one, but it looked like a nice machine and I was already curious about it back then. After struggling for a while to find a local laptop with a Ryzen CPU and that s nice and compact and that breaks the 16GB memory barrier, finding this one that jumped all the way to 64GB sealed the deal for me. This is the specs for the configuration I got:

• Ryzen 7 4800H 2.9GHz Octa Core CPU (4MB L2 cache, 8MB L3 cache, 7nm process).
• 64GB RAM (2x DDR4 2666mHz 32GB modules)
• 1TB nvme disk
• 14 1920 1280 (16:9 aspect ratio) matt display.
• Real ethernet port (gigabit)
• Intel Wifi 6 AX200 wireless ethernet
• Magnesium alloy chassis

This configuration cost R18 796 ( 947 / $1122). That s significantly cheaper than anything else I can get that even starts to approach these specs. So this is a cheap laptop, but you wouldn t think so by using it.

I used the Debian netinstall image to install, and installation was just another uneventful and boring Debian installation (yay!). Unfortunately it needs the firmware-iwlwifi and firmare-amd-graphics packages for the binary blobs that drives the wifi card and GPU. At least it works flawlessly and you don t need an additional non-free display driver (as is the case with NVidia GPUs). I haven t tested the graphics extensively yet, but desktop graphics performance is very snappy. This GPU also does fancy stuff like VP8/VP9 encoding/decoding, so I m curious to see how well it does next time I have to encode some videos. The wifi upgrade was nice for copying files over. My old laptop maxed out at 300Mbps, this one connects to my home network between 800-1000Mbps. At this speed I don t bother connecting via cable at home. I read on Twitter that Tuxedo Computers thinks that it s possible to bring Coreboot to this device. That would be yet another plus for this machine. I ll try to answer some of my own questions about this device that I had before, that other people in the Debian community might also have if they re interested in this device. Since many of us are familiar with the ThinkPad X200 series of laptops, I ll compare it a bit to my X250, and also a little to the X13 that I was considering before. Initially, I was a bit hesitant about the 14 form factor, since I really like the portability of the 12.5 ThinkPad. But because the screen bezel is a lot smaller, the Wootbook (that just rolls off the tongue a lot better than the PF4NU1F ) is just slightly wider than the X250. It weighs in at 1.1KG instead of the 1.38KG of the X250. It s also thinner, so even though it has a larger display, it actually feels a lot more portable. Here s a picture of my X250 on top of the Wootbook, you can see a few mm of Wootbook sticking out to the right.

Card Reader One thing that I overlooked when ordering this laptop was that it doesn t have an SD card reader. I see that some variations have them, like on this Slimbook review. It s not a deal-breaker for me, I have a USB card reader that s very light and that I ll just keep in my backpack. But if you re ordering one of these machines and have some choice, it might be something to look out for if it s something you care about. Keyboard/Touchpad On to the keyboard. This keyboard isn t quite as nice to type on as on the ThinkPad, but, it s not bad at all. I type on many different laptop keyboards and I would rank this keyboard very comfortably in the above average range. I ve been typing on it a lot over the last 3 days (including this blog post) and it started feeling natural very quickly and I m not distracted by it as much as I thought I would be transitioning from the ThinkPad or my mechanical desktop keyboard. In terms of layout, it s nice having an actual Insert button again. This is things normal users don t care about, but since I use mc (where insert selects files) this is a welcome return :). I also like that it doesn t have a Print Screen button at the bottom of my keyboard between alt and ctrl like the ThinkPad has. Unfortunately, it doesn t have dedicated pgup/pgdn buttons. I use those a lot in apps to switch between tabs. At leas the Fn button and the ctrl buttons are next to each other, so pressing those together with up and down to switch tabs isn t that horrible, but if I don t get used to it in another day or two I might do some remapping. The touchpad has en extra sensor-button on the top left corner that s used on Windows to temporarily disable the touchpad. I captured it s keyscan codes and it presses left control + keyscan code 93. The airplane mode, volume and brightness buttons work fine. I do miss the ThinkPad trackpoint. It s great especially in confined spaces, your hands don t have to move far from the keyboard for quick pointer operations and it s nice for doing something quick and accurate. I painted a bit in Krita last night, and agree with other reviewers that the touchpad could do with just a bit more resolution. I was initially disturbed when I noticed that my physical touchpad buttons were gone, but you get right-click by tapping with two fingers, and middle click with tapping 3 fingers. Not quite as efficient as having the real buttons, but it actually works ok. For the most part, this keyboard and touchpad is completely adequate. Only time will tell whether the keyboard still works fine in a few years from now, but I really have no serious complaints about it. Display The X250 had a brightness of 172 nits. That s not very bright, I think the X250 has about the dimmest display in the ThinkPad X200 range. This hasn t been a problem for me until recently, my eyes are very photo-sensitive so most of the time I use it at reduced brightness anyway, but since I ve been working from home a lot recently, it s nice to sometimes sit outside and work, especially now that it s spring time and we have some nice days. At full brightness, I can t see much on my X250 outside. The Wootbook is significantly brighter even (even at less than 50% brightness), although I couldn t find the exact specification for its brightness online. Ports The Wootbook has 3x USB type A ports and 1x USB type C port. That s already quite luxurious for a compact laptop. As I mentioned in the specs above, it also has a full-sized ethernet socket. On the new X13 (the new ThinkPad machine I was considering), you only get 2x USB type A ports and if you want ethernet, you have to buy an additional adapter that s quite expensive especially considering that it s just a cable adapter (I don t think it contains any electronics). It has one hdmi port. Initially I was a bit concerned at lack of displayport (which my X250 has), but with an adapter it s possible to convert the USB-C port to displayport and it seems like it s possible to connect up to 3 external displays without using something weird like display over usual USB3.

Overall remarks When maxing out the CPU, the fan is louder than on a ThinkPad, I definitely noticed it while compiling the zfs-dkms module. On the plus side, that happened incredibly fast. Comparing the Wootbook to my X250, the biggest downfall it has is really it s pointing device. It doesn t have a trackpad and the touchpad is ok and completely usable, but not great. I use my laptop on a desk most of the time so using an external mouse will mostly solve that. If money were no object, I would definitely choose a maxed out ThinkPad for its superior keyboard/mouse, but the X13 configured with 32GB of RAM and 128GB of SSD retails for just about double of what I paid for this machine. It doesn t seem like you can really buy the perfect laptop no matter how much money you want to spend, there s some compromise no matter what you end up choosing, but this machine packs quite a punch, especially for its price, and so far I m very happy with my purchase and the incredible performance it provides. I m also very glad that Wootware went with the gray/black colours, I prefer that by far to the white and silver variants. It s also the first laptop I ve had since 2006 that didn t come with Windows on it. The Wootbook is also comfortable/sturdy enough to carry with one hand while open. The ThinkPads are great like this and with many other brands this just feels unsafe. I don t feel as confident carrying it by it s display because it s very thin (I know, I shouldn t be doing that with the ThinkPads either, but I ve been doing that for years without a problem :) ). There s also a post on Reddit that tracks where you can buy these machines from various vendors all over the world.

Because posting private keys on the Internet is a bad idea, some people like to redact their private keys, so that it looks kinda-sorta like a private key, but it isn t actually giving away anything secret. Unfortunately, due to the way that private keys are represented, it is easy to redact a key in such a way that it doesn t actually redact anything at all. RSA private keys are particularly bad at this, but the problem can (potentially) apply to other keys as well. I ll show you a bit of Inside Baseball with key formats, and then demonstrate the practical implications. Finally, we ll go through a practical worked example from an actual not-really-redacted key I recently stumbled across in my travels.

The Private Lives of Private Keys Here is what a typical private key looks like, when you come across it:

-----BEGIN RSA PRIVATE KEY-----
MGICAQACEQCxjdTmecltJEz2PLMpS4BXAgMBAAECEDKtuwD17gpagnASq1zQTYEC
CQDVTYVsjjF7IQIJANUYZsIjRsR3AgkAkahDUXL0RSECCB78r2SnsJC9AghaOK3F
sKoELg==
-----END RSA PRIVATE KEY-----

Obviously, there s some hidden meaning in there computers don t encrypt things by shouting BEGIN RSA PRIVATE KEY! , after all. What is between the BEGIN/END lines above is, in fact, a base64-encoded DER format ASN.1 structure representing a PKCS#1 private key. In simple terms, it s a list of numbers very important numbers. The list of numbers is, in order:

• A version number (0);
• The public modulus , commonly referred to as n ;
• The public exponent , or e (which is almost always 65,537, for various unimportant reasons);
• The private exponent , or d ;
• The two private primes , or p and q ;
• Two exponents, which are known as dmp1 and dmq1 ; and
• A coefficient, known as iqmp .

Why Is This a Problem? The thing is, only three of those numbers are actually required in a private key. The rest, whilst useful to allow the RSA encryption and decryption to be more efficient, aren t necessary. The three absolutely required values are e, p, and q. Of the other numbers, most of them are at least about the same size as each of p and q. So of the total data in an RSA key, less than a quarter of the data is required. Let me show you with the above toy key, by breaking it down piece by piece¹:

• MGI DER for this is a sequence
• CAQ version (0)
• CxjdTmecltJEz2PLMpS4BX n
• AgMBAA e
• ECEDKtuwD17gpagnASq1zQTY d
• ECCQDVTYVsjjF7IQ p
• IJANUYZsIjRsR3 q
• AgkAkahDUXL0RS dmp1
• ECCB78r2SnsJC9 dmq1
• AghaOK3FsKoELg== iqmp

Remember that in order to reconstruct all of these values, all I need are e, p, and q and e is pretty much always 65,537. So I could redact almost all of this key, and still give all the important, private bits of this key. Let me show you:

-----BEGIN RSA PRIVATE KEY-----
..............................................................EC
CQDVTYVsjjF7IQIJANUYZsIjRsR3....................................
........
-----END RSA PRIVATE KEY-----

Now, I doubt that anyone is going to redact a key precisely like this but then again, this isn t a typical RSA key. They usually look a lot more like this:

-----BEGIN RSA PRIVATE KEY-----
MIIEogIBAAKCAQEAu6Inch7+mWtKn+leB9uCG3MaJIxRyvC/5KTz2fR+h+GOhqj4
SZJobiVB4FrE5FgC7AnlH6qeRi9MI0s6dt5UWZ5oNIeWSaOOeNO+EJDUkSVf67wj
SNGXlSjGAkPZ0nRJiDjhuPvQmdW53hOaBLk5udxPEQbenpXAzbLJ7wH5ouLQ3nQw
HwpwDNQhF6zRO8WoscpDVThOAM+s4PS7EiK8ZR4hu2toon8Ynadlm95V45wR0VlW
zywgbkZCKa1IMrDCscB6CglQ10M3Xzya3iTzDtQxYMVqhDrA7uBYRxA0y1sER+Rb
yhEh03xz3AWemJVLCQuU06r+FABXJuY/QuAVvQIDAQABAoIBAFqwWVhzWqNUlFEO
PoCVvCEAVRZtK+tmyZj9kU87ORz8DCNR8A+/T/JM17ZUqO2lDGSBs9jGYpGRsr8s
USm69BIM2ljpX95fyzDjRu5C0jsFUYNi/7rmctmJR4s4uENcKV5J/++k5oI0Jw4L
c1ntHNWUgjK8m0UTJIlHbQq0bbAoFEcfdZxd3W+SzRG3jND3gifqKxBG04YDwloy
tu+bPV2jEih6p8tykew5OJwtJ3XsSZnqJMwcvDciVbwYNiJ6pUvGq6Z9kumOavm9
XU26m4cWipuK0URWbHWQA7SjbktqEpxsFrn5bYhJ9qXgLUh/I1+WhB2GEf3hQF5A
pDTN4oECgYEA7Kp6lE7ugFBDC09sKAhoQWrVSiFpZG4Z1gsL9z5YmZU/vZf0Su0n
9J2/k5B1GghvSwkTqpDZLXgNz8eIX0WCsS1xpzOuORSNvS1DWuzyATIG2cExuRiB
jYWIJUeCpa5p2PdlZmBrnD/hJ4oNk4oAVpf+HisfDSN7HBpN+TJfcAUCgYEAyvY7
Y4hQfHIdcfF3A9eeCGazIYbwVyfoGu70S/BZb2NoNEPymqsz7NOfwZQkL4O7R3Wl
Rm0vrWT8T5ykEUgT+2ruZVXYSQCKUOl18acbAy0eZ81wGBljZc9VWBrP1rHviVWd
OVDRZNjz6nd6ZMrJvxRa24TvxZbJMmO1cgSW1FkCgYAoWBd1WM9HiGclcnCZknVT
UYbykCeLO0mkN1Xe2/32kH7BLzox26PIC2wxF5seyPlP7Ugw92hOW/zewsD4nLze
v0R0oFa+3EYdTa4BvgqzMXgBfvGfABJ1saG32SzoWYcpuWLLxPwTMsCLIPmXgRr1
qAtl0SwF7Vp7O/C23mNukQKBgB89DOEB7xloWv3Zo27U9f7nB7UmVsGjY8cZdkJl
6O4LB9PbjXCe3ywZWmJqEbO6e83A3sJbNdZjT65VNq9uP50X1T+FmfeKfL99X2jl
RnQTsrVZWmJrLfBSnBkmb0zlMDAcHEnhFYmHFuvEnfL7f1fIoz9cU6c+0RLPY/L7
n9dpAoGAXih17mcmtnV+Ce+lBWzGWw9P4kVDSIxzGxd8gprrGKLa3Q9VuOrLdt58
++UzNUaBN6VYAe4jgxGfZfh+IaSlMouwOjDgE/qzgY8QsjBubzmABR/KWCYiRqkj
qpWCgo1FC1Gn94gh/+dW2Q8+NjYtXWNqQcjRP4AKTBnPktEvdMA=
-----END RSA PRIVATE KEY-----

People typically redact keys by deleting whole lines, and usually replacing them with [...] and the like. But only about 345 of those 1588 characters (excluding the header and footer) are required to construct the entire key. You can redact about 4/5ths of that giant blob of stuff, and your private parts (or at least, those of your key) are still left uncomfortably exposed.

But Wait! There s More! Remember how I said that everything in the key other than e, p, and q could be derived from those three numbers? Let s talk about one of those numbers: n. This is known as the public modulus (because, along with e, it is also present in the public key). It is very easy to calculate: n = p * q. It is also very early in the key (the second number, in fact). Since n = p * q, it follows that q = n / p. Thus, as long as the key is intact up to p, you can derive q by simple division.

Real World Redaction At this point, I d like to introduce an acquaintance of mine: Mr. Johan Finn. He is the proud owner of the GitHub repo johanfinn/scripts. For a while, his repo contained a script that contained a poorly-redacted private key. He since deleted it, by making a new commit, but of course because git never really deletes anything, it s still available. Of course, Mr. Finn may delete the repo, or force-push a new history without that commit, so here is the redacted private key, with a bit of the surrounding shell script, for our illustrative pleasure:

#Add private key to .ssh folder
cd /home/johan/.ssh/
echo  "-----BEGIN RSA PRIVATE KEY-----
MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN
KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKK
 
MIIJKgIBAAKCAgEAxEVih1JGb8gu/Fm4AZh+ZwJw/pjzzliWrg4mICFt1g7SmIE2
TCQMKABdwd11wOFKCPc/UzRH/fHuQcvWrpbOSdqev/zKff9iedKw/YygkMeIRaXB
fYELqvUAOJ8PPfDm70st9GJRhjGgo5+L3cJB2gfgeiDNHzaFvapRSU0oMGQX+kI9
ezsjDAn+0Pp+r3h/u1QpLSH4moRFGF4omNydI+3iTGB98/EzuNhRBHRNq4oBV5SG
Pq/A1bem2ninnoEaQ+OPESxYzDz3Jy9jV0W/6LvtJ844m+XX69H5fqq5dy55z6DW
sGKn78ULPVZPsYH5Y7C+CM6GAn4nYCpau0t52sqsY5epXdeYx4Dc+Wm0CjXrUDEe
Egl4loPKDxJkQqQ/MQiz6Le/UK9vEmnWn1TRXK3ekzNV4NgDfJANBQobOpwt8WVB
rbsC0ON7n680RQnl7PltK9P1AQW5vHsahkoixk/BhcwhkrkZGyDIl9g8Q/Euyoq3
eivKPLz7/rhDE7C1BzFy7v8AjC3w7i9QeHcWOZFAXo5hiDasIAkljDOsdfD4tP5/
wSO6E6pjL3kJ+RH2FCHd7ciQb+IcuXbku64ln8gab4p8jLa/mcMI+V3eWYnZ82Yu
axsa85hAe4wb60cp/rCJo7ihhDTTvGooqtTisOv2nSvCYpcW9qbL6cGjAXECAwEA
AQKCAgEAjz6wnWDP5Y9ts2FrqUZ5ooamnzpUXlpLhrbu3m5ncl4ZF5LfH+QDN0Kl
KvONmHsUhJynC/vROybSJBU4Fu4bms1DJY3C39h/L7g00qhLG7901pgWMpn3QQtU
4P49qpBii20MGhuTsmQQALtV4kB/vTgYfinoawpo67cdYmk8lqzGzzB/HKxZdNTq
s+zOfxRr7PWMo9LyVRuKLjGyYXZJ/coFaobWBi8Y96Rw5NZZRYQQXLIalC/Dhndm
AHckpstEtx2i8f6yxEUOgPvV/gD7Akn92RpqOGW0g/kYpXjGqZQy9PVHGy61sInY
HSkcOspIkJiS6WyJY9JcvJPM6ns4b84GE9qoUlWVF3RWJk1dqYCw5hz4U8LFyxsF
R6WhYiImvjxBLpab55rSqbGkzjI2z+ucDZyl1gqIv9U6qceVsgRyuqdfVN4deU22
LzO5IEDhnGdFqg9KQY7u8zm686Ejs64T1sh0y4GOmGsSg+P6nsqkdlXH8C+Cf03F
lqPFg8WQC7ojl/S8dPmkT5tcJh3BPwIWuvbtVjFOGQc8x0lb+NwK8h2Nsn6LNazS
0H90adh/IyYX4sBMokrpxAi+gMAWiyJHIHLeH2itNKtAQd3qQowbrWNswJSgJzsT
JuJ7uqRKAFkE6nCeAkuj/6KHHMPsfCAffVdyGaWqhoxmPOrnVgECggEBAOrCCwiC
XxwUgjOfOKx68siFJLfHf4vPo42LZOkAQq5aUmcWHbJVXmoxLYSczyAROopY0wd6
Dx8rqnpO7OtZsdJMeBSHbMVKoBZ77hiCQlrljcj12moFaEAButLCdZFsZW4zF/sx
kWIAaPH9vc4MvHHyvyNoB3yQRdevu57X7xGf9UxWuPil/jvdbt9toaraUT6rUBWU
GYPNKaLFsQzKsFWAzp5RGpASkhuiBJ0Qx3cfLyirjrKqTipe3o3gh/5RSHQ6VAhz
gdUG7WszNWk8FDCL6RTWzPOrbUyJo/wz1kblsL3vhV7ldEKFHeEjsDGroW2VUFlS
asAHNvM4/uYcOSECggEBANYH0427qZtLVuL97htXW9kCAT75xbMwgRskAH4nJDlZ
IggDErmzBhtrHgR+9X09iL47jr7dUcrVNPHzK/WXALFSKzXhkG/yAgmt3r14WgJ6
5y7010LlPFrzaNEyO/S4ISuBLt4cinjJsrFpoo0WI8jXeM5ddG6ncxdurKXMymY7
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::.::
:::::::::::::::::::::::::::.::::::::::::::::::::::::::::::::::::
LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLlL
 
 
 
YYYYYYYYYYYYYYYYYYYYYyYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY
gff0GJCOMZ65pMSy3A3cSAtjlKnb4fWzuHD5CFbusN4WhCT/tNxGNSpzvxd8GIDs
nY7exs9L230oCCpedVgcbayHCbkChEfoPzL1e1jXjgCwCTgt8GjeEFqc1gXNEaUn
O8AJ4VlR8fRszHm6yR0ZUBdY7UJddxQiYOzt0S1RLlECggEAbdcs4mZdqf3OjejJ
06oTPs9NRtAJVZlppSi7pmmAyaNpOuKWMoLPElDAQ3Q7VX26LlExLCZoPOVpdqDH
KbdmBEfTR4e11Pn9vYdu9/i6o10U4hpmf4TYKlqk10g1Sj21l8JATj/7Diey8scO
sAI1iftSg3aBSj8W7rxCxSezrENzuqw5D95a/he1cMUTB6XuravqZK5O4eR0vrxR
AvMzXk5OXrUEALUvt84u6m6XZZ0pq5XZxq74s8p/x1JvTwcpJ3jDKNEixlHfdHEZ
ZIu/xpcwD5gRfVGQamdcWvzGHZYLBFO1y5kAtL8kI9tW7WaouWVLmv99AyxdAaCB
Y5mBAQKCAQEAzU7AnorPzYndlOzkxRFtp6MGsvRBsvvqPLCyUFEXrHNV872O7tdO
GmsMZl+q+TJXw7O54FjJJvqSSS1sk68AGRirHop7VQce8U36BmI2ZX6j2SVAgIkI
9m3btCCt5rfiCatn2+Qg6HECmrCsHw6H0RbwaXS4RZUXD/k4X+sslBitOb7K+Y+N
Bacq6QxxjlIqQdKKPs4P2PNHEAey+kEJJGEQ7bTkNxCZ21kgi1Sc5L8U/IGy0BMC
PvJxssLdaWILyp3Ws8Q4RAoC5c0ZP0W2j+5NSbi3jsDFi0Y6/2GRdY1HAZX4twem
Q0NCedq1JNatP1gsb6bcnVHFDEGsj/35oQKCAQEAgmWMuSrojR/fjJzvke6Wvbox
FRnPk+6YRzuYhAP/YPxSRYyB5at++5Q1qr7QWn7NFozFIVFFT8CBU36ktWQ39MGm
cJ5SGyN9nAbbuWA6e+/u059R7QL+6f64xHRAGyLT3gOb1G0N6h7VqFT25q5Tq0rc
Lf/CvLKoudjv+sQ5GKBPT18+zxmwJ8YUWAsXUyrqoFWY/Tvo5yLxaC0W2gh3+Ppi
EDqe4RRJ3VKuKfZxHn5VLxgtBFN96Gy0+Htm5tiMKOZMYAkHiL+vrVZAX0hIEuRZ
JJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJ
MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
-----END RSA PRIVATE KEY-----" >> id_rsa

Now, if you try to reconstruct this key by removing the obvious garbage lines (the ones that are all repeated characters, some of which aren t even valid base64 characters), it still isn t a key at least, openssl pkey doesn t want anything to do with it. The key is very much still in there, though, as we shall soon see. Using a gem I wrote and a quick bit of Ruby, we can extract a complete private key. The irb session looks something like this:

>> require "derparse"
>> b64 = <<EOF
MIIJKgIBAAKCAgEAxEVih1JGb8gu/Fm4AZh+ZwJw/pjzzliWrg4mICFt1g7SmIE2
TCQMKABdwd11wOFKCPc/UzRH/fHuQcvWrpbOSdqev/zKff9iedKw/YygkMeIRaXB
fYELqvUAOJ8PPfDm70st9GJRhjGgo5+L3cJB2gfgeiDNHzaFvapRSU0oMGQX+kI9
ezsjDAn+0Pp+r3h/u1QpLSH4moRFGF4omNydI+3iTGB98/EzuNhRBHRNq4oBV5SG
Pq/A1bem2ninnoEaQ+OPESxYzDz3Jy9jV0W/6LvtJ844m+XX69H5fqq5dy55z6DW
sGKn78ULPVZPsYH5Y7C+CM6GAn4nYCpau0t52sqsY5epXdeYx4Dc+Wm0CjXrUDEe
Egl4loPKDxJkQqQ/MQiz6Le/UK9vEmnWn1TRXK3ekzNV4NgDfJANBQobOpwt8WVB
rbsC0ON7n680RQnl7PltK9P1AQW5vHsahkoixk/BhcwhkrkZGyDIl9g8Q/Euyoq3
eivKPLz7/rhDE7C1BzFy7v8AjC3w7i9QeHcWOZFAXo5hiDasIAkljDOsdfD4tP5/
wSO6E6pjL3kJ+RH2FCHd7ciQb+IcuXbku64ln8gab4p8jLa/mcMI+V3eWYnZ82Yu
axsa85hAe4wb60cp/rCJo7ihhDTTvGooqtTisOv2nSvCYpcW9qbL6cGjAXECAwEA
AQKCAgEAjz6wnWDP5Y9ts2FrqUZ5ooamnzpUXlpLhrbu3m5ncl4ZF5LfH+QDN0Kl
KvONmHsUhJynC/vROybSJBU4Fu4bms1DJY3C39h/L7g00qhLG7901pgWMpn3QQtU
4P49qpBii20MGhuTsmQQALtV4kB/vTgYfinoawpo67cdYmk8lqzGzzB/HKxZdNTq
s+zOfxRr7PWMo9LyVRuKLjGyYXZJ/coFaobWBi8Y96Rw5NZZRYQQXLIalC/Dhndm
AHckpstEtx2i8f6yxEUOgPvV/gD7Akn92RpqOGW0g/kYpXjGqZQy9PVHGy61sInY
HSkcOspIkJiS6WyJY9JcvJPM6ns4b84GE9qoUlWVF3RWJk1dqYCw5hz4U8LFyxsF
R6WhYiImvjxBLpab55rSqbGkzjI2z+ucDZyl1gqIv9U6qceVsgRyuqdfVN4deU22
LzO5IEDhnGdFqg9KQY7u8zm686Ejs64T1sh0y4GOmGsSg+P6nsqkdlXH8C+Cf03F
lqPFg8WQC7ojl/S8dPmkT5tcJh3BPwIWuvbtVjFOGQc8x0lb+NwK8h2Nsn6LNazS
0H90adh/IyYX4sBMokrpxAi+gMAWiyJHIHLeH2itNKtAQd3qQowbrWNswJSgJzsT
JuJ7uqRKAFkE6nCeAkuj/6KHHMPsfCAffVdyGaWqhoxmPOrnVgECggEBAOrCCwiC
XxwUgjOfOKx68siFJLfHf4vPo42LZOkAQq5aUmcWHbJVXmoxLYSczyAROopY0wd6
Dx8rqnpO7OtZsdJMeBSHbMVKoBZ77hiCQlrljcj12moFaEAButLCdZFsZW4zF/sx
kWIAaPH9vc4MvHHyvyNoB3yQRdevu57X7xGf9UxWuPil/jvdbt9toaraUT6rUBWU
GYPNKaLFsQzKsFWAzp5RGpASkhuiBJ0Qx3cfLyirjrKqTipe3o3gh/5RSHQ6VAhz
gdUG7WszNWk8FDCL6RTWzPOrbUyJo/wz1kblsL3vhV7ldEKFHeEjsDGroW2VUFlS
asAHNvM4/uYcOSECggEBANYH0427qZtLVuL97htXW9kCAT75xbMwgRskAH4nJDlZ
IggDErmzBhtrHgR+9X09iL47jr7dUcrVNPHzK/WXALFSKzXhkG/yAgmt3r14WgJ6
5y7010LlPFrzaNEyO/S4ISuBLt4cinjJsrFpoo0WI8jXeM5ddG6ncxdurKXMymY7
EOF
>> b64 += <<EOF
gff0GJCOMZ65pMSy3A3cSAtjlKnb4fWzuHD5CFbusN4WhCT/tNxGNSpzvxd8GIDs
nY7exs9L230oCCpedVgcbayHCbkChEfoPzL1e1jXjgCwCTgt8GjeEFqc1gXNEaUn
O8AJ4VlR8fRszHm6yR0ZUBdY7UJddxQiYOzt0S1RLlECggEAbdcs4mZdqf3OjejJ
06oTPs9NRtAJVZlppSi7pmmAyaNpOuKWMoLPElDAQ3Q7VX26LlExLCZoPOVpdqDH
KbdmBEfTR4e11Pn9vYdu9/i6o10U4hpmf4TYKlqk10g1Sj21l8JATj/7Diey8scO
sAI1iftSg3aBSj8W7rxCxSezrENzuqw5D95a/he1cMUTB6XuravqZK5O4eR0vrxR
AvMzXk5OXrUEALUvt84u6m6XZZ0pq5XZxq74s8p/x1JvTwcpJ3jDKNEixlHfdHEZ
ZIu/xpcwD5gRfVGQamdcWvzGHZYLBFO1y5kAtL8kI9tW7WaouWVLmv99AyxdAaCB
Y5mBAQKCAQEAzU7AnorPzYndlOzkxRFtp6MGsvRBsvvqPLCyUFEXrHNV872O7tdO
GmsMZl+q+TJXw7O54FjJJvqSSS1sk68AGRirHop7VQce8U36BmI2ZX6j2SVAgIkI
9m3btCCt5rfiCatn2+Qg6HECmrCsHw6H0RbwaXS4RZUXD/k4X+sslBitOb7K+Y+N
Bacq6QxxjlIqQdKKPs4P2PNHEAey+kEJJGEQ7bTkNxCZ21kgi1Sc5L8U/IGy0BMC
PvJxssLdaWILyp3Ws8Q4RAoC5c0ZP0W2j+5NSbi3jsDFi0Y6/2GRdY1HAZX4twem
Q0NCedq1JNatP1gsb6bcnVHFDEGsj/35oQKCAQEAgmWMuSrojR/fjJzvke6Wvbox
FRnPk+6YRzuYhAP/YPxSRYyB5at++5Q1qr7QWn7NFozFIVFFT8CBU36ktWQ39MGm
cJ5SGyN9nAbbuWA6e+/u059R7QL+6f64xHRAGyLT3gOb1G0N6h7VqFT25q5Tq0rc
Lf/CvLKoudjv+sQ5GKBPT18+zxmwJ8YUWAsXUyrqoFWY/Tvo5yLxaC0W2gh3+Ppi
EDqe4RRJ3VKuKfZxHn5VLxgtBFN96Gy0+Htm5tiMKOZMYAkHiL+vrVZAX0hIEuRZ
EOF
>> der = b64.unpack("m").first
>> c = DerParse.new(der).first_node.first_child
>> version = c.value
=> 0
>> c = c.next_node
>> n = c.value
=> 80071596234464993385068908004931... # (etc)
>> c = c.next_node
>> e = c.value
=> 65537
>> c = c.next_node
>> d = c.value
=> 58438813486895877116761996105770... # (etc)
>> c = c.next_node
>> p = c.value
=> 29635449580247160226960937109864... # (etc)
>> c = c.next_node
>> q = c.value
=> 27018856595256414771163410576410... # (etc)

What I ve done, in case you don t speak Ruby, is take the two chunks of plausible-looking base64 data, chuck them together into a variable named b64, unbase64 it into a variable named der, pass that into a new DerParse instance, and then walk the DER value tree until I got all the values I need. Interestingly, the q value actually traverses the split in the two chunks, which means that there s always the possibility that there are lines missing from the key. However, since p and q are supposed to be prime, we can sanity check them to see if corruption is likely to have occurred:

>> require "openssl"
>> OpenSSL::BN.new(p).prime?
=> true
>> OpenSSL::BN.new(q).prime?
=> true

Excellent! The chances of a corrupted file producing valid-but-incorrect prime numbers isn t huge, so we can be fairly confident that we ve got the real p and q. Now, with the help of another one of my creations we can use e, p, and q to create a fully-operational battle key:

>> require "openssl/pkey/rsa"
>> k = OpenSSL::PKey::RSA.from_factors(p, q, e)
=> #<OpenSSL::PKey::RSA:0x0000559d5903cd38>
>> k.valid?
=> true
>> k.verify(OpenSSL::Digest::SHA256.new, k.sign(OpenSSL::Digest::SHA256.new, "bob"), "bob")
=> true

and there you have it. One fairly redacted-looking private key brought back to life by maths and far too much free time. Sorry Mr. Finn, I hope you re not still using that key on anything Internet-facing.

What About Other Key Types? EC keys are very different beasts, but they have much the same problems as RSA keys. A typical EC key contains both private and public data, and the public portion is twice the size so only about 1/3 of the data in the key is private material. It is quite plausible that you can redact an EC key and leave all the actually private bits exposed.

What Do We Do About It? In short: don t ever try and redact real private keys. For documentation purposes, just put KEY GOES HERE in the appropriate spot, or something like that. Store your secrets somewhere that isn t a public (or even private!) git repo. Generating a dummy private key and sticking it in there isn t a great idea, for different reasons: people have this odd habit of reusing demo keys in real life. There s no need to encourage that sort of thing.

---

1. Technically the pieces aren t 100% aligned with the underlying DER, because of how base64 works. I felt it was easier to understand if I stuck to chopping up the base64, rather than decoding into DER and then chopping up the DER.

The well-known kill system call has been around for decades and is used to send a signal to another process. The most common use is to terminate or kill another process by sending the KILL or TERM signal but it can be used for a form of IPC, usually around giving the other process a kick to do something. One thing that isn t as well known is besides sending a signal to a process, you can send some data to it. This can either be an integer or a pointer and uses similar semantics to the known kill and signal handler. I came across this when there was a merge request for procps. The main changes are using sigqueue instead of kill in the sender and using a signal action not a signal handler in the receiver. To illustrate this feature, I have a small set of programs called sender and receiver that will pass an integer between them. The Sender The sender program is extremely simple, use a random(ish) from time masked to two bytes, put it in the required union and send the lot to sendqueue.

# include <signal.h>
# include <stdlib.h>
# include <stdio.h>
# include <time.h>
int main(int argc, char **argv)
 
    union sigval sigval;
    pid_t pid;
    if (argc < 2   (pid = atoi(argv[1])) < 0)
	return EXIT_FAILURE;
    sigval.sival_int = time(NULL) &amp; 0xff;
    printf("sender: sending %d to PID %d\n",
        sigval.sival_int, pid);
    sigqueue(pid, SIGUSR1, sigval);
    return EXIT_SUCCESS;
 

The key lines are 13 and 16 where the random (ish) integer is stored in the sigval union and then sent to the other process with the sigqueue. The receiver The receiver just sets up the signal handler, sends its PID (so I know what to tell the sender) and sits in a sleeping loop.

# include <stdio.h>
# include <stdlib.h>
# include <sys/types.h>
# include <unistd.h>
# include <signal.h>
void signal_handler(int signum, siginfo_t *siginfo, void *ucontext)
 
    if (signum != SIGUSR1) return;
    if (siginfo->si_code != SI_QUEUE) return;
    printf("receiver: Got value %d\n",
	    siginfo->si_int);
 
int main(int argc, char **argv)
 
    pid_t pid = getpid();
    struct sigaction signal_action;
    printf("receiver: PID is %d\n", pid);
    signal_action.sa_sigaction = signal_handler;
    sigemptyset (&amp;signal_action.sa_mask);
    signal_action.sa_flags = SA_SIGINFO;
    sigaction(SIGUSR1, &amp;signal_action, NULL);
    while(1) sleep(100);
    return EXIT_SUCCESS;
 

Lines 16-26 setup the signal handler. The main difference here is SA_SIGINFO used for the signal flags and sigaction references a sa_sigaction function rather than sa_handler. We need to use a different function because the sigaction only is passed the signal number but we need more information, including the integer that the sender process stored in sigval. Lines 7-14 are the signal handler function itself. It first checks that the receiver process got the correct signal (SIGUSR1 in this case) and that we got this signal from sigqueue because the type is SI_QUEUE. Checking the type of signal is important because different signals give you different data. For example, if you signalled this process with kill then si_int is undefined. The result As a proof of concept, the results are not terribly exciting. We see the sender say what it will be sending and the receiver saying it got it. It was useful to get some results, especially when things went wrong.

$ ./receiver &
[1] 370216
receiver: PID is 370216
$ ./sender 370216
sender: sending 133 to (gdPID 370216
receiver: Got value 133

Gotchas While testing the two process there was two gotchas I encountered. GDB and the siginfo structure The sigaction manual page shows a simple siginfo_t structure, however when looking at what is passed to the signal handler, it s much more complicated.

(gdb) p *siginfo
$2 =  si_signo = 10, si_errno = 0, si_code = -1, __pad0 = 0, _sifields =  _pad =  371539, 1000, 11, 32766, 0 <repeats 24 times> , _kill =  si_pid = 371539, si_uid = 1000 , _timer =  si_tid = 371539, 
      si_overrun = 1000, si_sigval =  sival_int = 11, sival_ptr = 0x7ffe0000000b , _rt =  si_pid = 371539, si_uid = 1000, si_sigval =  sival_int = 11, sival_ptr = 0x7ffe0000000b , _sigchld =  
      si_pid = 371539, si_uid = 1000, si_status = 11, si_utime = 0, si_stime = 0 , _sigfault =  si_addr = 0x3e80005ab53, si_addr_lsb = 11, _bounds =  _addr_bnd =  _lower = 0x0, _upper = 0x0 , _pkey = 0 , 
    _sigpoll =  si_band = 4294967667539, si_fd = 11 , _sigsys =  _call_addr = 0x3e80005ab53, _syscall = 11, _arch = 32766 
(gdb) p siginfo->_sifields._rt.si_sigval.sival_int
$3 = 11

So the integer is stored in a union in a structure in a structure. Much harder to find than just simply sival_int. The pointer is just a pointer So perhaps sending an integer is not enough. The sigval is a union with an integer and a pointer. Could a string be sent instead? I changed line 13 of the sender so it used a string instead of an integer.

    sigval.sival_ptr = "Hello, World!"

The receiver needed a minor adjustment to print out the string. I tried this and the receiver segmentation faulted. What was going on? The issue is the set of system calls does a simple passing of the values. So if the sender sends a pointer to a string located at 0x1234567 then the receiver will have a pointer to the same location. When the receiver tries to dereference the sival_ptr, it is pointing to memory that is not owned by it but by another process (the sender) so it segmentation faults. The solution would be to use shared memory between the processes. The signal queue would then use the pointer to the shared memory and, in theory, all would be well.

Welcome to gambaru.de. Here is my monthly report that covers what I have been doing for Debian. If you re interested in Java, Games and LTS topics, this might be interesting for you. Debian Games

• I packaged a new upstream version of springlobby. There is even a more recent one now but I discovered that it would fail to build from source. I reported the issue and now I am waiting for another release.
• These packages were also updated: bullet, tuxfootball (#876481), berusky (#877979), spring, hitori and trackballs.
• I released a new version of cube2-data, a DFSG-free version of the Sauerbraten game. This release was largely made possible thanks to the work of Nyav.
• I prepared two stable point releases of berusky and simutrans to fix #877979 and # 869029 for users of Debian s stable distributions too. The bug in Berusky is already resolved but I m still waiting for the confirmation to upload simutrans (#878668).
• I updated wing and biniax2. Here I discovered that biniax2 would segfault immediately at startup after recompilation. I tracked down the issue to some C code that caused undefined behavior, prepared a patch and released a fixed revision.
• I sponsored a new upstream version of mupen64plus-qt.

Debian Java

• This month I started to work on fixing Java9 bugs since Java 9 shall become the new default JDK/JRE for Buster. The bug reports were filed by Chris West who did the important work of identifying build failures and broken packages. I started with some low hanging fruits first and the following packages are now Java 9 ready: libgetopt-java, libjide-oss-java, activemq-protobuf, antelope, yecht, slashtime, colorpicker, f2j, libreadline-java, libjaxp1.3-java, jlapack, isorelax, libisrt-java, rxtx, uima-addons.
• New upstream releases this month: apktool, jboss-xnio, okio, pdfsam, libsejda-java, bcel, autocomplete, mediathekview, sweethome3d.
• MediathekView introduced yet another build-dependency. Let s welcome libokhttp-java in Debian.
• I upgraded jackson-databind to fix CVE-2017-7525. While I was at it, I continued this work with jackson-core, jackson-annotations, jackson-dataformat-xml, jackson-jr, jackson-datatype-joda, jackson-module-jaxb-annotations, jackson-dataformat-cbor, jackson-dataformat-smile, jackson-dataformat-yaml and jackson-jaxrs-providers. I also requested the removal of jackson-datatype-guava.
• More resolved RC issues: commons-io (#873118), tycho (#879250)
• Package updates: mockobjects (converted from CDBS to DH) and jblas (RC #877225, #873212, #698176)
• The Maven 2 to Maven 3 transition caused (and still causes) a lot of fallout: I investigated the following packages with RC bugs. In most cases the issue was in another package, so the bugs could be closed but there were also packages like conversant-disruptor (#869002) which caused build failures unrelated to the transition. In total 15 packages were triaged or fixed: jasypt (#871195), mustache-java (#869009), libslf4j-java, apache-log4j2, conversant-disruptor, powermock(#869017), jetty9(#869021), maven-site-plugin(#869001), javamail(#871102), assertj-core(#871131), java-allocation-instrumenter(#869251), json-smart(#868603), sisu-guice(#868611), maven-archiver(#871069), doxia-sitetools(#875948)
• I have started to work on a new upstream version of triplea, multiple strategy games written in Java. The update would fix a couple of bugs and make the package ready for Java 9.
• It was also requested to upgrade Gradle to version 3.4.1 at least. I have made good progress but there is more work to do.

Debian LTS This was my twentieth month as a paid contributor and I have been paid to work 19 hours on Debian LTS, a project started by Rapha l Hertzog. I will catch up with the remaining 1,75 hours in November. In that time I did the following:

• From 30. October to 05. November I was in charge of our LTS frontdesk. I triaged bugs in jasperreports, jbossas4, libstruts1.2-java, httpcomponents-client, vim, emacs23, trafficserver, async-http-client, liblouis, wordpress, apr, apr-utils, redis, nautilus, libpam4j and spip.
• I decided to mark jbossas4 as end-of-life because the Java application server was never fully packaged and the version in Wheezy is already nine years old. I investigated the open security issues in jasperreports and contacted upstream but they have not published any details yet.
• I pinged bug #878088. The reportbug maintainer still has to respond to the idea of informing the security teams when users report bugs in security uploads. I will discuss the possibility with the rest of the team, whether it is helpful to patch reportbug in Wheezy/Jessie/Stretch now.
• DLA-1151-1 and DLA-1160-1. Issued two security updates for WordPress addressing 10 CVE. It was later discovered that the patch for CVE-2017-14990 was incomplete and caused a regression when using WordPress multi-site feature. Single-site installations were not affected. The complete fix would either include a database upgrade or a different approach without using the new database field signup_id . I reverted the patch for now and issued a regression update in DLA-1151-2.
• DLA-1158-1. Issued a security update for bchunk fixing 3 CVE.
• DLA-1159-1. Issued a security update for graphicsmagick fixing 2 CVE.
• DLA-1164-1. Issued a security update for mupdf fixing 2 CVE.
• DLA-1165-1. Issued a security update for libpam4j fixing 1 CVE.
• DLA-1167-1. Issued a security update for ruby-yajl fixing 1 CVE.
• DLA-1157-1. I uploaded a security update for openssl. The update was prepared by Kurt Roeckx, the maintainer of openssl.

Misc

• I prepared the security updates for libpam4j (DSA-4025-1) and bchunk (DSA-4026-1) and fixed the same issues in Sid and Buster.

Thanks for reading and see you next time.

I said that I was going to start writing these regularly, so I'm going to stick to it, even when the results are rather underwhelming. One of the goals is to make the time for more free software work, and I do better at doing things that I record. The only piece of free software work for September was that I made rra-c-util compile cleanly with the Clang static analyzer. This was fairly tedious work that mostly involved unconfusing the compiler or converting (semi-intentional) crashes into explicit asserts, but it unblocks using the Clang static analyzer as part of the automated test suite of my other projects that are downstream of rra-c-util. One of the semantic changes I made was that the vector utilities in rra-c-util (which maintain a resizable array of strings) now always allocate room for at least one string pointer. This wastes a small amount of memory for empty vectors that are never used, but ensures that the strings struct member is always valid. This isn't, strictly speaking, a correctness fix, since all the checks were correct, but after some thought, I decided that humans might have the same problem that the static analyzer had. It's a lot easier to reason about a field that's never NULL. Similarly, the replacement function for a missing reallocarray now does an allocation of size 1 if given a size of 0, just to avoid edge case behavior. (I'm sure the behavior of a realloc with size 0 is defined somewhere in the C standard, but if I have to look it up, I'd rather not make a human reason about it.) I started on, but didn't finish, making rra-c-util compile without Clang warnings (at least for a chosen set of warnings). By far the hardest problem here are the Clang warnings for comparisons between unsigned and signed integers. In theory, I like this warning, since it's the cause of a lot of very obscure bugs. In practice, gah does C ever do this all over the place, and it's incredibly painful to avoid. (One of the biggest offenders is write, which returns a ssize_t that you almost always want to compare against a size_t.) I did a bunch of mechanical work, but I now have a lot of bits of code like:

     if (status < 0)
         return;
    written = (size_t) status;
    if (written < avail)
        buffer->left += written;

which is ugly and unsatisfying. And I also have a ton of casts, such as with:

    buffer_resize(buffer, (size_t) st.st_size + used);

since st.st_size is an off_t, which may be signed. This is all deeply unsatisfying and ugly, and I think it makes the code moderately harder to read, but I do think the warning will potentially catch bugs and even security issues. I'm still torn. Maybe I can find some nice macros or programming styles to avoid the worst of this problem. It definitely requires more thought, rather than just committing this huge mechanical change with lots of ugly code. Mostly, this kind of nonsense makes me want to stop working on C code and go finish learning Rust.... Anyway, apart from work, the biggest thing I managed to do last month that was vaguely related to free software was upgrading my personal servers to stretch (finally). That mostly went okay; only a few things made it unnecessarily exciting. The first was that one of my systems had a very tiny / partition that was too small to hold the downloaded debs for the upgrade, so I had to resize it (VM disk, partition, and file system), and that was a bit exciting because it has an old-style DOS partition table that isn't aligned (hmmm, which is probably why disk I/O is so slow on those VMs), so I had to use the obsolete fdisk -c=dos mode because I wasn't up for replacing the partition right then. The second was that my first try at an upgrade died with a segfault during the libc6 postinst and then every executable segfaulted. A mild panic and a rescue disk later (and thirty minutes and a lot of swearing), I tracked the problem down to libc6-xen. Nothing in the dependency structure between jessie and stretch forces libc6-xen to be upgraded in lockstep or removed, but it's earlier in the search path. So ld.so gets upgraded, and then finds the old libc6 from the libc6-xen package, and the mismatch causes immediate segfaults. A chroot dpkg --purge from the rescue disk solved the problem as soon as I knew what was going on, but that was a stressful half-hour. The third problem was something I should have known was going to be an issue: an old Perl program that does some internal stuff for one of the services I ran had a defined @array test that has been warning for eons and that I never fixed. That became a full syntax error with the most recent Perl, and then I fixed it incorrectly the first time and had a bunch of trouble tracking down what I'd broken. All sorted out now, and everything is happily running stretch. (ejabberd, which other folks had mentioned was a problem, went completely smoothly, although I suspect I now have too many of the plugin packages installed and should do a purging.)

Changes

• myrepos: mention related software
• icns: OpenJPEG 2 fixes (1 2 3 4), fix minor buffer overrun
• check-all-the-things: fix cwd usage, dependencies updates (1, 2), increased verbosity, TODO items (for errcheck, coccinelle, DR.CHECKER, RATS, qmllint, xmlpatternsvalidator, gfapy-validate, phpstan, psalm, ansible-lint)
• spelling dictionaries:
  • codespell: delte, defualt, Debiab, aiffer, syntesis, reprociblbe, thether, suport, mimimum, isssue, reseting
  • lintian: delte, Debiab, aiffer, syntesis, reprociblbe
• Debian Planets: add BCCD, Pardus
• Debian derivatives census: improve db handling
• Debian PTS: fix backports issue
• Debian member portfolio: https (1, 2, 3), drop alioth files, fix duck & lists search
• Debian security tracker: BlueBourne NFUs, u-boot issues, kanboard CVE list fixes, fix version typo
• Debian puppet: SSL exception (1, 2), comment fixes
• Debian website: add missing section title
• Debian wiki pages: AutomaticPackagingTools, CrossCompiling, DebianDevelopment, DebianLive/buster, DebianYeeloong/ko, Derivatives/Census (QA, BCCD, EmmabuntusDE (1, 2), ev3dev, Netrunner (1, 2), Pardus), DeveloperNews, Diagrams, EmDebian/CrossDebootstrap, InstallingDebianOn/Asus/Transformer T300 Chi/Stretch, IRC, JamesAmontgomery, JensKorte/reportbug-by-mail, LocalGroupsTemplate, Mobile, MultimediaFetchingTools, OfflineMasterKey (1, 2), Outreachy/Round15 (Projects/SocialEventAndConferenceCalendars), PaulWise/InterestingSoftware, PortTemplate, qa.debian.org, research/publications, RISC-V, Sprints/2017/DebianCloudOct2017, SystemBuildTools, Teams (Dpkg/FAQ, Publicity/micronews, ReleaseTeam/ReleaseCheckList), template
• File Formats wiki pages: ZED (1, 2)

Issues

• Crashes in firefox-esr, lugaru, minetest-mod-animals
• Conffile removal in fonts-beng-extra
• Upstream renamed openggsn
• Strictness in ftp.d.o
• Features in unattended-upgrades
• Missing hashes in caffeine-developers PPA
• Missing conffile removal in di-netboot-assistant
• Incorrect upgrades in unattended-upgrades
• Incorrect URL in reproducible-check
• Verbosity in github-backup
• Minor issues in unattended-upgrades
• Usability issue in Firefox
• Upstreaming needed for Hone Linux-Sensor
• New upstream version of samba

Review

• Spam: reported 1 LWN comment, 5 Debian bug reports and 338 Debian mailing list posts
• Debian packages: sponsored minetest-mod-unifieddyes 20170925-1
• Debian wiki: RecentChanges for the month
• Debian screenshots:
  • approved banshee, blender, bsdgames, chromium, eficas, elpa-visual-fill-column, elpa-writegood-mode, glirc, gnome-builder, gnome-shell-extension-taskbar, gwenview, knocker, krita, ksysguard, masscan, mediainfo-gui, mousepad, mysql-workbench, nitrogen, ola, pavucontrol, qupzilla, tiger, transmission-gtk, virtualbox, vlc, wavemon, wfuzz
  • deleted mysql-workbench (ugly, Windows), udisks2 (of other software)
  • rejected elpa-* (MacOS and taken from upstream), 0ad (RPi logo), synaptic (weird overlaid stuff), dehydrated (hilarious but not a screenshot), di-netboot-assistant (of other software), wfuzz (help output)
  • approved deletion of udisks2 (of other software), scottfree (manual page), agenda.app (of other software)
  • rejected deletion of reportbug/abe/nlkt (bogus reason)

Administration

• icns: merged patches
• Debian: help guest user with access, investigate/escalate broken network, restart broken stunnels, investigate static.d.o storage, investigate weird RAID mails, ask hoster to investigate power issue,
• Debian mentors: lintian/security updates & reboot
• Debian wiki: merged & deployed patch, redirect DDTSS translator, redirect user support requests, whitelist email addresses, update email for accounts with bouncing email,
• Debian derivatives census: merged/deployed patches
• Debian PTS: debugged cron mails, deployed changes, reran scripts, fixed configuration file
• Openmoko: debug reboot issue, debug load issues

Communication

• Inform derivatives that are using Alioth about the pending shutdown
• Inquire about Netrunner, Pardus in the Debian derivatives census.
• Invite OB2D Linux to the Debian derivatives census
• Welcome Rohan Garg (of Netrunner), Aaron Weeden (of BCCD), Muhammet Kara (of Pardus) to the Debian derivatives census

Sponsors The samba bug was sponsored by my employer. All other work was done on a volunteer basis.

mksh R56 was released with experimental fixes for the history no longer persisted when HISTFILE near-full and interactive shell cannot wait on coprocess by PID issues (I hope they do not introduce any regressioins) and otherwise as a bugfix release. You might wish to know the $EDITOR selection mechanism in dot.mkshrc changed. Some more alias characters are allowed again, and POSIX character classes (for ASCII, and EBCDIC, only) appeared by popular vote. mksh now has a FAQ; enjoy. Do feel free to contribute (answers, too, of course). The jupp text editor has also received a new release; asides from being much smaller, and updated (mksh too, btw) to Unicode 10, and some segfault fixes, it features falling back to using /dev/tty if stdin or stdout is not a terminal (for use on GNU with find xargs jupp, since they don t have our xargs(1) -o option yet), a new command to exit nonzero (sometimes, utilities invoking the generic visual editor need this), and presentation mode . Presentation mode, crediting Natureshadow, is basically putting your slides as (UTF-8, with fancy stuff inside) plaintext files into one directory, with sorting names (so e.g. zero-padded slide numbers as filenames), presenting them with jupp * in a fullscreen xterm. You d hit F6 to switch to one-file view first, then present by using F8 to go forward (F7 to go backward), and, for demonstrations, F9 to pipe the entire slide through an external command (could be just sh ) offering the previous one as default. Simple yet powerful; I imagine Sven Guckes would love it, were he not such a vim user. The new release is offered as source tarball (as usual) and in distribution packages, but also, again, a Win32 version as PKZIP archive (right-click on setup.inf and hit I nstall to install it). Note that this comes with its own (thankfully local) version of the Cygwin32 library (compatible down to Windows 95, apparently), so if you have Cygwin installed yourself you re better off compiling it there and using your own version instead. I ve also released a new DOS version of 2.8 with no code patches but an updated jupprc; the binary (self-extracting LHarc archive) this time comes with all resource files, not just jupp s. Today, the jupprc drop-in file for JOE 3.7 got a matching update (and some fixes for bugs discovered during that) and I added a new one for JOE 4.4 (the former being in Debian wheezy, the latter in jessie, stretch and buster/sid). It s a bit rudimentary (the new shell window functionality is absent) but, mostly, gives the desired jupp feeling, more so than just using stock jstar would.

• source tarballs
• Win32 binaries
• DOS binaries
• drop-in jupprc for JOE 2.8, or to update jupp 2.8
• drop-in jupprc for JOE 3.7
• drop-in jupprc for JOE 4.4

CVS ability to commit to multiple branches of a file at the same time, therefore grouping the commit (by commitid at least, unsure if cvsps et al. can be persuaded to recognise it). If you don t know what cvs(GNU) is: it is a proper (although not distributed) version control system and the best for centralised tasks. (For decentral tasks, abusing git as pseudo-VCS has won by popularity vote; take this as a comparison.) If desired, I can make these new versions available in my WTF APT repository on request. (Debian buster/sid users: please change https to http there, the site is only available with TLSv1.0 as it doesn t require bank-level security.) I d welcome it very much if people using an OS which does not yet carry either to package it there. Message me when one more is added, too In unrelated news I uploaded MuseScore 2.1 to Debian unstable, mostly because the maintainers are busy (though I could comaintain it if needed, I d just need help with the C++ and CMake details). Bonus side effect is that I can now build 2.2~ test versions with patches of mine added I plan to produce to fix some issues (and submit upstream) (read more )

Changes

• apt: more flexible timing (1, 2, 3), /org to /srv
• apt-offline: remove generated files
• check-all-the-things: add more shell checks
• spelling dictionaries:
  • codespell: bettery, ciruit(s), debugginf, treshold, actuion(able), apendix, behaivior, confimred, meaningfull, registeres, sence, systen, tryed, infrasctructure, current(ly), defaut, pedning, variabele, upstreamedd, nework, ouput
  • lintian: resistence, bettery, ciruit(s), debugginf, actuion, actuionable, apendix, behaivior, confimred, finaly, meaningfull, registeres, sence, systen, tryed, infrasctructure, currnt(ly), defaut, pedning, variabele, upstreamedd
• Debian security tracker: add gsoap CVE-2017-9765
• Debian puppet: stretch fixes (1, 2), failed attempt to kill a mail flood, disable web cache backups
• Debian QA services: email update, https for lintian.d.o, fix issue with by-hash
• Debian build log scanner: https, html log links, ignore uploaders brokenness
• Debian Planets: https for Grml, add Zevenet, Deepin, remove HandyLinux
• Debian website: use ISO dates, no https for archive.d.o
• Debian package uploads: flasm 1.62-8
• Debian wiki pages: DebianDay/2017/Venezuela/Caracas, Derivatives/Census (Template, Deepin, Grml, HandyLinux, Proxmox, SELKS, xanadu, Zevenet), Hardware/Wanted, Infinity0, IRC, JavierFernandezSanguino, LocalGroups/Debian-CN, LTS/Team, MemberBenefits, Mobile, PaulWise/InterestingSoftware, qa.debian.org, ReleasePartyStretch/India/Hyderabad/MGIT, SystemBuildTools
• DebConf wiki pages: DebConf17/DebconfShirts

Issues

• Crashes in MoinMoin, aspell
• Missing sources in apt-offline (reported privately)
• Malformed data in ftp.d.o
• Debug symbols in perl
• Media playback in gstreamer
• Race condition in dehydrated
• Weird errors in translate-shell

Review

• Spam: reported several Debian bug reports and 359 Debian mailing list posts
• Debian wiki: RecentChanges for the month
• Debian screenshots:
  • approved baresip, blender, caja-dropbox, cups, fastnetmon, firetools, fonts-firacode, freecad, gemrb-baldurs-gate-2, gjiten, gscan2pdf, kdevelop, ketm, liguidsoap, lordsawar, mdadm, minizinc-ide, mupdf, musique, nautilus-dropbox, pcsxr, projectl, quiterss, rafkill, redis-server, redshift-gtk, residualvm, sigil, sqlitebrowser, surf, trigger-rally, virtualbox, vis, vizigrep, vokoscreen, xinv3d, yarssr
  • approved deletion of phototonic (Windows screenshot), pipebench (inappropriate background), autossh/gtk3-nocsd (not the package)
  • rejected deletion of muon-discover (we do not remove old screenshots), apt (not a bug reporting site)

Administration

• Debian: fsck/reboot a buildd, reboot a segfaulting buildd, report/fix broken hoster contact, ping hoster about down machines, forcibly reset backup machine, merged cache patch for network-test.d.o, do some samhain dances, fix two stunnel services, update an IP address in LDAP, fix /etc/aliases on one host, reboot 1 non-responsive VM
• Debian mentors: security updates, reboot
• Debian wiki: whitelist several email addresses
• Debian build log scanner: deploy my changes
• Debian PTS: deploy my changes
• Openmoko: security updates & reboots

Communication

• Ping Advogato users on Planet Debian about updating/removing their feeds since it shut down
• Invite deepin to the Debian derivatives census
• Welcome Deepin to the Debian derivatives census
• Inquire about the status of GreenboneOS, HandyLinux

Sponsors All work was done on a volunteer basis.

Debian Long Term Support (LTS) This is my monthly working on Debian LTS. This time I worked on various hairy issues surrounding ca-certificates, unattended-upgrades, apache2 regressions, libmtp, tcpdump and ipsec-tools.

ca-certificates updates I've been working on the removal of the Wosign and StartCom certificates (Debian bug #858539) and, in general, the synchronisation of ca-certificates across suites (Debian bug #867461) since at least last march. I have made an attempt at summarizing the issue which led to a productive discussion and it seems that, in the end, the maintainer will take care of synchronizing information across suites. Guido was right in again raising the question of synchronizing NSS across all suites (Debian bug #824872) which itself raised the other question of how to test reverse dependencies. This brings me back to Debian bug #817286 which, basically proposed the idea of having "proposed updates" for security issues. The problem is while we can upload test packages to stable proposed-updates, we can't do the same in LTS because the suite is closed and we operate only on security packages. This issue came up before in other security upload and we need to think better about how to solve this.

unattended-upgrades Speaking of security upgrades brings me to the question of a bug (Debian bug #867169) that was filed against the wheezy version of unattended-upgrades, which showed that the package simply stopped working since the latest stable release, because wheezy became "oldoldstable". I first suggested using the "codename" but that appears to have been introduced only after wheezy. In the end, I proposed a simple update that would fix the configuration files and uploaded this as DLA-1032-1. This is thankfully fixed in later releases and will not require such hackery when jessie becomes LTS as well.

libmtp Next up is the work on the libmtp vulnerabilities (CVE-2017-9831 and CVE-2017-9832). As I described in my announcement, the work to backport the patch was huge, as upstream basically backported a whole library from the gphoto2 package to fix those issues (and probably many more). The lack of a test suite made it difficult to trust my own work, but given that I had no (negative) feedback, I figured it was okay to simply upload the result and that became DLA-1029-1.

tcpdump I then looked at reproducing CVE-2017-11108, a heap overflow triggered tcpdump would parse specifically STP packets. In Debian bug #867718, I described how to reproduce the issue across all suites and opened an issue upstream, given that the upstream maintainers hadn't responded responded in weeks according to notes in the RedHat Bugzilla issue. I eventually worked on a patch which I shared upstream, but that was rejected as they were already working on it in their embargoed repository. I can explain this confusion and duplication of work with:

1. the original submitter didn't really contact security@tcpdump.org
2. he did and they didn't reply, being just too busy
3. they replied and he didn't relay that information back

I think #2 is most likely: the tcpdump.org folks are probably very busy with tons of reports like this. Still, I should probably have contacted security@tcpdump.org directly before starting my work, even though no harm was done because I didn't divulge issues that were already public. Since then, tcpdump has released 4.9.1 which fixes the issue, but then new CVEs came out that will require more work and probably another release. People looking into this issue must be certain to coordinate with the tcpdump security team before fixing the actual issues.

ipsec-tools Another package that didn't quite have a working solution is the ipsec-tools suite, in which the racoon daemon was vulnerable to a remotely-triggered DOS attack (CVE-2016-10396). I reviewed and fixed the upstream patch which introduced a regression. Unfortunately, there is no test suite or proof of concept to control the results. The reality is that ipsec-tools is really old, and should maybe simply be removed from Debian, in favor of strongswan. Upstream hasn't done a release in years and various distributions have patched up forks of those to keep it alive... I was happy, however, to know that a maintainer will take care of updating the various suites, including LTS, with my improved patch. So this fixes the issue for now, but I would strongly encourage users to switch away from ipsec-tools in the future.

apache2 Finally, I was bitten by the old DLA-841-1 upload I did all the way back in February, as it introduced a regression (Debian bug #858373). It turns out it was possible to segfault Apache workers with a trivial HTTP request, in certain (rather exotic, I might add) configurations (ErrorDocument 400 directive pointing to a cgid script in worker mode). Still, it was a serious regression and I found a part of the nasty long patch we worked on back then that was faulty, and introduced a small fix to correct that. The proposed package unfortunately didn't yield any feedback, and I can only assume it will work okay for people. The result is the DLA-841-2 upload which fixes the regression. I unfortunately didn't have time to work on the remaining CVEs affecting apache2 in LTS at the time of writing.

Triage I also did some miscellaneous triage by filing Debian bug #867477 for poppler in an effort to document better the pending issue. Next up was some minor work on eglibc issues. CVE-2017-8804 has a patch, but it's been disputed. since the main victim of this and the core of the vulnerability (rpcbind) has already been fixed, I am not sure this vulnerability is still a thing in LTS at all. I also looked at CVE-2014-9984, but the code is so different in wheezy that I wonder if LTS is affected at all. Unfortunately, the eglibc gymnastics are a little beyond me and I do not feel confident enough to just push those issues aside for now and let them open for others to look at.

Other free software work And of course, there's my usual monthly volunteer work. My ratio is a little better this time, having reached an about even ratio between paid and volunteer work, whereas this was 60% volunteer work in march.

Announcing ecdysis I recently published ecdysis, a set of template and code samples that I frequently reuse across project. This is probably the least pronounceable project name I have ever chosen, but this is somewhat on purpose. The goal of this project is not collaboration or to become a library: it's just a personal project which I share with the world as a curiosity. To quote the README file:

The name comes from what snakes and other animals do to "create a new snake": they shed their skin. This is not so appropriate for snakes, as it's just a way to rejuvenate their skin, but is especially relevant for anthropods since then "ecdysis" may be associated with a metamorphosis:

Ecdysis is the moulting of the cuticle in many invertebrates of the clade Ecdysozoa. Since the cuticle of these animals typically forms a largely inelastic exoskeleton, it is shed during growth and a new, larger covering is formed. The remnants of the old, empty exoskeleton are called exuviae. Wikipedia

So this project is metamorphosed into others when the documentation templates, code examples and so on are reused elsewhere. For that reason, the license is an unusally liberal (for me) MIT/Expat license. The name also has the nice property of being absolutely unpronounceable, which makes it unlikely to be copied but easy to search online.

It was an interesting exercise to go back into older projects and factor out interesting code. The process is not complete yet, as there are older projects I'm still curious in reviewing. A bunch of that code could also be factored into upstream project and maybe even the Python standard library. In short, this is stuff I keep on forgetting how to do: a proper setup.py config, some fancy argparse extensions and so on. Instead of having to remember where I had written that clever piece of code, I now shove it in the crazy chaotic project where I can find it again in the future.

Beets experiments Since I started using Subsonic (or Libresonic) to manage the music on my phone, album covers are suddenly way more interesting. But my collection so far has had limited album covers: my other media player (gmpc) would download those on the fly on its own and store them in its own database - not on the filesystem. I guess this could be considered to be a limitation of Subsonic, but I actually appreciate the separation of duty here. Garbage in, garbage out: the quality of Subsonic's rendering depends largely on how well setup your library and tags are. It turns out there is an amazing tool called beets to do exactly that kind of stuff. I originally discarded that "media library management system for obsessive-compulsive [OC] music geeks", trying to convince myself i was not an "OC music geek". Turns out I am. Oh well. Thanks to beets, I was able to download album covers for a lot of the albums in my collection. The only covers that are missing now are albums that are not correctly tagged and that beets couldn't automatically fix up. I still need to go through those and fix all those tags, but the first run did an impressive job at getting album covers. Then I got the next crazy idea: after a camping trip where we forgot (again) the lyrics to Georges Brassens, I figured I could start putting some lyrics on my ebook reader. "How hard can that be?" of course, being the start of another crazy project. A pull request and 3 days later, I had something that could turn a beets lyrics database into a Sphinx document which, in turn, can be turned into an ePUB. In the process, I probably got blocked from MusixMatch a hundred times, but it's done. Phew! The resulting e-book is about 8000 pages long, but is still surprisingly responsive. In the process, I also happened to do a partial benchmark of Python's bloom filter libraries. The biggest surprise there was the performance of the set builtin: for small items, it is basically as fast as a bloom filter. Of course, when the item size grows larger, its memory usage explodes, but in this case it turned out to be sufficient and bloom filter completely overkill and confusing. Oh, and thanks to those efforts, I got admitted in the beetbox organization on GitHub! I am not sure what I will do with that newfound power: I was just scratching an itch, really. But hopefully I'll be able to help here and there in the future as well.

Debian package maintenance I did some normal upkeep on a bunch of my packages this month, that were long overdue:

• uploaded slop 6.3.47-1: major new upstream release
• uploaded an NMU for maim 5.4.64-1.1: maim was broken by the slop release
• uploaded pv 1.6.6-1: new upstream release
• uploaded kedpm 1.0+deb8u1 to jessie (oldstable): one last security fix (Debian bug #860817, CVE-2017-8296) for that derelict password manager
• uploaded charybdis 3.5.5-1: new minor upstream release, with optional support for mbedtls
• filed Debian bug #866786 against cryptsetup to make the remote initramfs SSH-based unlocking support multiple devices: thanks to the maintainer, this now works flawlessly in buster and may be backported to stretch
• expanded on Debian bug #805414 against gdm3 and Debian bug #845938 against pulseaudio, because I had trouble connecting my computer to this new Bluetooth speaker. turns out this is a known issue in Pulseaudio: whereas it releases ALSA devices, it doesn't release Bluetooth devices properly. Documented this more clearly in the wiki page
• filed Debian bug #866790 regarding old stray Apparmor profiles that were lying around my system after an upgrade, which got me interested in Debian bug #830502 in turn
• filed Debian bug #868728 against cups regarding a weird behavior I had interacting with a network printer. turns out the other workstation was misconfigured... why are printers still so hard?
• filed Debian bug #870102 to automate sbuild schroots upgrades
• after playing around with rash tried to complete the packaging (Debian bug #754972) of percol with this pull request upstream. this ended up to be way too much overhead and I reverted to my old normal history habits.

This morning I remembered I had a beefy virtual-server setup to run some kernel builds upon (when I was playing with Linux security moduels), and I figured before I shut it down I should use the power to run some fuzzing. As I was writing some code in Emacs at the time I figured "why not fuzz emacs?" After a few hours this was the result:

 deagol ~ $ perl -e 'print " " x ( 1024 * 1024  * 12);' > t.el
 deagol ~ $ /usr/bin/emacs --batch --script ./t.el
 ..
 ..
 Segmentation fault (core dumped)

Yup, evaluating an lisp file caused a segfault, due to a stack-overflow (no security implications). I've never been more proud, even though I have contributed code to GNU Emacs in the past.

Here's what happened in the Reproducible Builds effort between Sunday May 28 and Saturday June 3 2017: Past an upcoming events

• On June 9th, Chris Lamb will present at the Hong Kong Open Source Conference 2017 on reproducible builds.
• We restarted our IRC meetings, now with a monthly schedule where we meet on the first Thursday of the month at a varying time. Our next meeting will be on the first 6th of July at 17:00 UTC. At the June meeting we had the following agenda, and if you are interested there is an automated summary and full logs too.
  1. Introductions
  2. Feedback for the reproducible.json spec format - is that suitable for Guix and F-Droid as well? LEDE/OpenWrt? Coreboot?
  3. writing parser + user interface for reproducible.json
  4. tests.r-b.o/Debian once Stretch has been released
  5. Reproducible Builds Summit 2017
  6. Next meeting
  7. Any other business

Documentation updates

• Holger added a link to the blog post with a report about the recent hackathon to our website.

Toolchain development and fixes

• Chris Lamb wrote a proof-of-concept implementation for #863622 ("apt: warn when installing packages that are not reproducible"). Try it, it's fun!
• Russ worked on making the debian-policy package build reproducibly which is a good opportunity to shamelessly plug that #844431 ("debian-policy: Packages should be reproducible") is still a work in progress.

Patches and bugs filed

• Bernhard M. Wiedemann:
  • pycryptopp
  • rpm
  • rpm
  • bam
  • boost/jam
  • premake
  • ruby/rdoc
  • ruby/rdoc
  • sdcc (merged)

4 package reviews have been added, 6 have been updated and 25 have been removed in this week, adding to our knowledge about identified issues. Weekly QA work During our reproducibility testing, FTBFS bugs have been detected and reported by:

• Adrian Bunk (2)
• Chris Lamb (1)

diffoscope development

• Chris Lamb:
  • Add comparator for Fontconfig cache files.
• anthraxx:
  • tools: extend external Arch Linux tools
• Ximin Luo:
  • Add a --exclude-command CLI for filtering out long-running commands like "readelf --debug-dump=info"
  • Use unicode chars for the +/- controls instead of hacky punctuation
  • Don't show +/- controls for differences with no children
  • Fix create_limited_print_func
  • Add a size() method to Difference and check that self._visuals is empty in get_reverse()
  • Add a reader for the JSON format
  • Allow the "source" param to overridden compare() methods to be given as a positional argument

tests.reproducible-builds.org Mattia Rizzolo:

• packages-tests: go back to the shorter way of specifying JUnit jobs.
• update_jdn: call jenkins-jobs instead of jenkins-job-builder, as that's how it is named in 1.6.1 version.
• Upload jenkins-job-builder 1.6.1-1~bpo8+1 to jessie-backports.

Daniel Kahn Gillmor:

• Contribute a few typo fixes.

Vagrant Cascadian:

• Update README to reflect switch to PostgreSQL (some time ago).
• Add three new boards for armhf tests:
  • ff64a-armhf-rb.debian.net: Firefly-rk3399, Rockchip six-core (Cortex-A72 x2 + Cortex-A53 x4), 2GB RAM, USB-sata (future plans for native sata.)
  • jtx1a-armhf-rb.debian.net: Jetson-tx1, quad-core (big.LITTLE Cortex-A53/A57), ~3.5GB RAM, native SATA ~500GB disk
  • odc2a-armhf-rb.debian.net: Odroic-C2, quad-core (Cortex-A53), ~2GB ram, 128GB eMMC
• Ressurect rpi2c-armhf-rb.debian.net (Raspberry PI 2B, broadcom bcm2836 quad-core (cortex-A7), 1GB RAM) from the dead.

Holger Levsen:

• Configure the rc.local service on all build nodes to only start when the network is actually up, as configuring half of them to run in the future requires networking to determine the real current date. See Running Services After the Network is up if you want to learn more about networking with systemd. Somehow this doesnt work yet on the Ubuntu 16.05 arm64 nodes; help welcome, either on #debian-qa or #debian-reproducible.
• Add rpi2c back to the armhf network.
• Also add ff64a, odc2a & jtx1a to the armhf network.
  • Add pbuilder/schroot-setup jobs for the new nodes.
  • Add 10 new armhf builder jobs.
  • Disable all build jobs on odc2a again as haveged segfaults on 4.12~rc2, which is needed for this board & setup.
• Reproducible_cleanup_nodes.sh: adapt for new build service
• Make performance.html show if build jobs are down due to remote node problems and make code to count enabled jobs more robust.
• reproducible_build_service: Make it cope with disabled workers.
• jenkins-master-wrapper: Fail loudly if remote node is not accessable.
• Increase maximum scheduling queue sizes, to adjust for faster scheduler.
• New script, init_node, to initialize new build nodes (derived from refactored existing script).
• Update INSTALL documentation to reflect that we're testing arm64 now too.

Misc. This week's edition was written by Chris Lamb, Bernhard M. Wiedemann and Holger Levsen & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Here is my monthly update covering what I have been doing in the free software world (previous month):

• Wrote and released installation-birthday. Installing this package will celebrate the anniversary of installing your system by sending you an email via cron(8).
• Fixed an issue in the Django web development framework where you couldn't run the testsuite against a read-only copy of the source code. This was found by the Debian Continuous Integration service. (#26755)
• Provided a pull request for the "wammu" mobile phone manager to ensure the build is reproducible. (#49)

---

My monthly report covers a large part of what I have been doing in the free software world. I write it for my donors (thanks to them!) but also for the wider Debian community because it can give ideas to newcomers and it s one of the best ways to find volunteers to work with me on projects that matter to me. Debian LTS I was allocated 10 hours to work on security updates for Debian 7 Wheezy and had 1.5 hours remaining from March. During this time I did the following:

• I released DLA-905-1 on ghostscript fixing 3 CVE. I also triaged two other ghostscript CVE that were not relevant to the version in wheezy.
• I started to look into CVE-2016-10209 for libarchive but was not able to reproduce the segfault and marked it as not worth an update (same decision as security team).
• After many tries to get more details from upstream of libxml-twig-perl on CVE-2016-9180, I decided that the low severity of the issue was not worth spending more time on it (same decision as RedHat and Debian security team).
• I released DLA-921-1 on slurm-llnl fixing 1 high-severity CVE.
• I investigated CVE-2016-8686 on potrace and marked it as not requiring an update because the impact is very low. I documented the fact that it s fixed in unstable and asked the upstream author for the specific patch (no answer yet though).

Kali and pkg-security I updated the britney instance that we are using in Kali and spotted two small documentation mistakes that I fixed. We had a long-standing bug in Kali where extensions would stay visible on the lock screen. It was hard to reproduce and this month we finally managed to nail down the conditions required to reproduce it. It turns out that EasyScreenCast was the culprit. We paid Emilio Pozuelo Monfort to work on a patch and he fixed the problem in EasyScreenCast and also in gnome-shell, as a buggy extension should not have resulted in this behavior. I responded to multiple queries of new contributors in the pkg-security team. The team is rather active and it would be great if we could have a few more Debian developers to help review and sponsor the work our enthusiastic new members. Thanks See you next month for a new summary of my activities. Hopefully, I will be more active again between kids vacations, French elections and Zelda Breadth of the Wild, I got very much distracted from Debian last month.

One comment Liked this article? Click here. My blog is Flattr-enabled.

Changes

• myrepos: webcheckout considers https URLs as anonymous, switch URLs from http to https and fall back on terminal tools when X11 is gone
• fdupes: NUL separator option
• whohas: add more TODO sites
• apt: more flexibility in APT::Periodic interval options
• foxtrotgps: fix maps-for-free.com, httpsify website links (1, 2) and remove website comment
• lm-sensors: workaround RRD format inflexibility
• check-all-the-things: terminal fixes (1, 2, 3), Perl fixes (1, 2, 3, 4, 5, 6), grep fixes, more key checks, wrapping fixes (1, 2), release (1, 2) and TODO items for chap/Devel::Plumber
• youtube-dl: video downloader list
• devscripts: add support for running aptitude to chdist, add support for cgit URLs to debcheckout and fix issues when $HOME is unset (1, 2)
• Spelling:
  • codespell: sort dictionary, explot(ing), menue(s), mimimise, intall, cas, overrided, icluding, pleaes, interract(ing), interracts, deffered, upsteram, componet(s), depracated, annoncement, settting, pasword, post-morten, converion, verifyied and achive
  • lintian: explot(ing), menue(s), mimimise, intall, cas, icluding, interract(ing), interracts, deffered, upsteram, componet(s), depracated, annoncement, adminstration, inappropiate, pasword, post-morten and verifyied
• Debian wiki: Andrewsomething/Fonts/PackagingPolicy, AptitudeTodo, AudioVideo, BogdanPurcareata/PluggableAptBackend, BTS/NewcomerTag, DebianColombia (Eventos, MiniDebconf2017), DebianEdu/Status/Stretch, DebianEvents/br, (2017/MiniDebconfCuritiba), Debian_GNU/kFreeBSD/Jessie, DebianIndia/PackagingSessions, DebianInstaller/Qemu, DebianKernel/UserspaceTools, DebianMentorsFaq, DebianOnHandhelds/OpenPandora, DebianPackage (es, fr, ru, sv), DebianReleaseFAQ, DebianRepository (Setup), DebianScience/Debci, DebianSingleSignOn, DebianWomen/Projects/NewbieTasks, Derivatives/Census (QA, AstraLinux, CumulusLinux, CumulusLinux, Kali, Netrunner, Parsix, Tanglu, TurnKeyLinux, Ubuntu, Wazo), FreedomBox/LeavingTheCloud, fr (genisoimage, Quota), hal, Hardware/Wanted, HelpDebian (FAQ (it), TODO), how-can-i-help, (Ideas, fr), HP/ProLiant, Init, (fr), Initrd, (fr), InstallingDebianOn, (InstallingDebianOn/HP/EliteBook 820 G1, Razer/BladeStealth, it/Init, it/Initrd, Mempo/mempo-deb/tar, motd, (it, fr), MulheresBrasil (1, 2), MultimediaFetchingTools (1, 2), Packaging/BinaryPackage, PaulWise/InterestingSoftware, ReproducibleBuilds (BuildinfoInfrastructure, History), ru/ReposRelease, SecureApt, ServicesSSL (1, 2), SHA-1, Statistics, SuitesAndReposExtension, SystemBuildTools, Teams (Publicity, ReleaseTeam/ReleaseCheckList), UntrustedDebs, VideoDownloaders and WebBrowsers
• Debian wiki theme: remove last editor info
• Debian website: fix encoding issues, fix year issues and fix DPL update issue (1, 2)
• Debian mirrors: fix encoding issues
• Debian security tracker: encuentro embeds youtube-dl and add mediawiki issues (1, 2)
• Debian derivatives census: workaround apt issue with stretch, reduce max patch size to 5 MiB, document how to get large patches, ignore safe debian/changelog symlinks and add FIXMEs
• Debian Planets: Add Wazo, BunsenLabs RSS feed update, https for blog feeds and new Cumulus Linux/Netrunner logos
• Debian package uploads: python-debianbts, poppler, cruft, clamav-unofficial-sigs, config-package-dev, libarchive, talloc, tevent, libconfig-crontab-perl 1.42-1~bpo7+1 & 1.42-1~bpo8+1, openchange and check-all-the-things
• Debian puppet: switch from deprecated function, drop custom update-ca-certificates on stretch (1, 2) and support syslog-ng 3.8
• Debian meta-packages: www.d.o stretch updates and qa.d.o vcswatch deps
• Debian LDAP UI: use TLS for direct LDAP access
• Debian publicity: prefer verification to SHA-1
• Debian release: add latest two release updates
• Debian PTS: switch from gift to newcomer tags

Issues

• Crashes in remmina, liferea (1, 2) and diffoscope
• Non-distributable files in Ubuntu
• Debian testing migration unblocks of check-all-the-things
• Debian Policy violation in muttdown
• Move crash-whitepaper from Debian non-free to main
• Normal issues in needrestart (1, 2), liferea and git-crecord
• Features in needrestart, github-backup, systemctl, android-permissions, reprepro, freerdp, git-crecord (1, 2, 3), Debian manual pages, Debian sources browser, Debian Maintainer Dashboard and dman
• New versions of pyvmomi
• Typos in fenrir
• Code copy in encuentro
• Race conditions in git-hub
• Terminal handling in needrestart
• Bogus home directory handling in how-can-i-help, ruby, bash, dash and busybox sh
• CodePlex closing checks for lintian and duck
• Missing hashes in Ubuntu
• Weird spaces in evolution
• Verbosity in apt
• Security issue in nagstamon and a CSS injection attack elsewhere
• Minor issues in reportbug and debsources (1, 2)

Review

• Spam: reported 5 Debian bug reports and 246 Debian mailing list posts
• Debian wiki: RecentChanges for the month
• Debian packages: reviewed openscap-daemon, gsignond, qt5ct and sponsored open-ath9k-htc-firmware
• Debian screenshots:
  • approved 4pane, amiwm, btrfs-heatmap, cenon.app, desmume, fonts-atarismall, fonts-goudybookletter, fonts-junicode, fonts-ocr-a, fonts-ocr-b, gnome-paint, grafx2, higan (2), lm-sensors, lshw-gtk, mediathekview, mrboom, munin-plugins-btrfs, nano, openlugaru, project-x, protracker, software-properties-gtk (2), termit, torbrowser-launcher, wmaker, w-scan, caveexpress (2), telegram-desktop (2), gtimelog, taglog, gtimer, osmose-emulator (2), rox-filer, fonts-blankenburg, topcat, balsa (2), gnome-multi-writer (4), colorize, wine, mate-menu and mate-tweak
  • deleted osmose-emulator (6 screenshots of non-free games)
  • rejected xserver-xorg-video-amdgpu (drivers have no UI), eom (includes image of offensive historical figure), synaptic (includes non-free background image), osmose-emulator (6 screenshots of non-free games) and libomxil-bellagio0-components-xvideo (borked upload)
  • approved deletion of xserver-xephyr (includes image of non-free OS), perforate (image of the homepage) and openvas/lv2core (not screenshots)
  • rejected deletion of vkeybd (junk in reason field)

Administration

• Debian systems: quiet a logrotate warning, investigate issue with DNSSEC and alioth, deploy fix on our first stretch buildd, restore alioth git repo after history rewrite, investigate iptables segfaults on buildd and investigate time issues on a NAS
• Debian derivatives census: delete patches over 5 MiB, re-enable the service
• Debian wiki: investigate some 403 errors, fix alioth KGB config, deploy theme changes, close a bogus bug report, ping 1 user with bouncing email, whitelist 9 email addresses and whitelist 2 domains
• Debian QA: deploy my changes
• Debian mentors: security upgrades and service restarts
• Openmoko: debug mailing list issue, security upgrades and reboots

Communication

• Invite Wazo to the Debian derivatives census
• Welcome ubilinux, Wazo and Roopa Prabhu (of Cumulus Linux) to the Debian derivatives census
• Discuss HP/ProLiant wiki page with HPE folks
• Inform git history rewriter about the git mailmap feature

Sponsors The libconfig-crontab-perl backports and pyvmomi issue were sponsored by my employer. All other work was done on a volunteer basis.

Here is my monthly update covering what I have been doing in the free software world (previous month):

• I was elected Debian Project Leader for 2017. I'd like to sincerely thank everyone who voted for me as well as everyone who took part in the election in general especially Mehdi Dogguy for being a worthy opponent. The result was covered on LWN, Phoronix, DistroWatch, iTWire, etc.
• Added support for the Monzo banking API in social-core, a Python library to allow web applications to authenticate using third-parties. (#68)
• Fixed a HTML injection attack in a demo of Russell Keith-Magee's BeeWare presentation library. (#3)
• Updated systemd's documentation to explain why we suggest explicitly calling make all despite the Makefile's "check" target calling it. (#5830)
• Updated the documentation of a breadth-first version of find(1) called bfs to refer to the newly-uploaded Debian package. (#23)
• Updated the configuration for the ticketbot IRC bot (zwiebelbot on OFTC) to identify #reproducible-builds as a Debian-related channel. This is so that bug Debian bug numbers are automatically expanded by the bot. (#7)

---

I recently diagnosed a problem in Debian's pam-p11 package. This package allegedly permits logging into a computer using a smart card or USB security token containing an ssh key. If you know the PIN and have the token, then your login attempt is authorized against the ssh authorized keys file. This seems like a great way to permit console logins as root to machines without having a shared password. Unfortunately, the package didn't work very well for me. It worked once, then all future attempts to use it segfaulted. I'm familiar with how PAM works. I understand the basic ideas behind PKCS11 (the API used for this type of smart card), but was completely unfamiliar with this particular PAM module and the PKCS11 library it used. The segfault was in an area of code I didn't even expect that this PAM module would ever call. Back in 1994, that would have been a painful slog. Gdb has improved significantly since then, and I'd really like to thank all the people over the years who made that possible. I was able to isolate the problem in just a couple of hours of debugging. Here are some of the cool features I used:

• "target record-full" which allows you to track what's going on so you can go backwards and potentially bisect where in a running program something goes wrong. It's not perfect; it seems to have trouble with memset and a few other functions, but it's really good.
• Hardware watch points. Once you know what memory is getting clobbered, have the hardware report all changes so you can see who's responsible.
• Hey, wait, what? I really wish I had placed a breakpoint back there. With "target record-full" and "reverse-continue," you can. Set the breakpoint and then reverse continue, and time runs backwards until your breakpoint gets hit.
• I didn't need it for this session, but "set follow-fork-mode" is very handy for certain applications. There's even a way to debug both the parent and child of a fork at the same time, although I always have to go look up the syntax. It seems like it ought to be "set follow-fork-mode both," and there was once a debugger that used that syntax, but Gdb uses different syntax for the same concept.

Anyway, with just a couple of hours and no instrumentation of the code, I managed to track down how a bunch of structures were being freed as an unexpected side effect of one of the function calls. Neither I nor the author of the pam-p11 module expected that (although it is documented and does make sense in retrospect). Good tools make life easier.

Just a short PSA for those around working with F5 devices: TMOS 11.6 introduced an experimental "mv" command in tmsh. In the last days we tried it for the first time on TMOS 12.1.1. It worked fine for a VirtualServer but a mv for a pool caused a sefault in tmm. We're currently working with the F5 support to sort it out, they think it's a known issue. Recommendation for now is to not use mv on pools. Just do it the old way, create a new pool, assign the new pool to the relevant VS and delete the old pool. Possible bug ID at F5 is ID562808. Since I can not find it in the TMOS 12.2 release notes I expect that this issue also applies to TMOS 12.2, but I did not verify that.

Multithreading continues to be hard (although the alternatives are not really a lot better). While debugging a user issue in Nageru, I found and fixed a few races (mostly harmless in practice, though) in my own code, but also two issues that I filed patches for in Mesa. But that's not enough, it seems; there are still issues that are too subtle for me to figure out on-the-fly. But at least with those patches, I can use interlaced video sources in Nageru on Intel GPUs without segfaulting pretty much immediately. My laptop's GPU isn't fast enough to actually run the YADIF interlacer realtime in 1080p60, though, but it's nice at least not take the program down. (These things are super-sensitive to timing, of course, which is probably why I didn't see them when developing the feature a year or so ago.) As usual, NVIDIA's proprietary drivers seem to be near-flawless in this regard. I'm starting to think maybe it's about massive amounts of QA resources.

Welcome to gambaru.de. Here is my monthly report that covers what I have been doing for Debian. If you re interested in Android, Java, Games and LTS topics, this might be interesting for you. Debian Android

• I sponsored and reviewed android-platform-external-libunwind and android-platform-frameworks-base for Kai-Chung. Two more packages, android-framework-23 and android-platform-tools-swt, are currently waiting in the NEW queue and hopefully they become available soon. Kai-Chung also became Debian Maintainer this month and I granted him upload permissions for android-platform-tools-base for a start. Congratulations.
• I packaged a new upstream release of apktool (2.2.1).

Debian Games

• I fixed RC bugs in lordsawar (#839323) and doomsday (#839338).
• I packaged new upstream releases of atanks, lordsawar, blockattack and peg-e.
• I completed the Bullet transition (#839243). Bullet 2.85 has also been released this month but it is now too late for Stretch because the transition freeze is already on the 5th of November. I expect more point releases a la 2.85.x during the coming weeks and I intend to provide an updated package in experimental soon.
• I did some cleanups, package upgrades and bug fixes for box2d and redeclipse (apparently redeclipse-server requires the -data package to be present now).
• I uploaded Redeclipse 1.5.6 to jessie-backports in the hope that more players will be able to connect to the multiplayer servers. Unfortunately network compatibility breaks rather frequently.
• I applied a patch from Gianfranco Costamagna to address an Multiarch installation issue (#841824) in FreeOrion.

Debian Java

• This month I tended to upgrade more Java packages including new upstream releases of libjide-oss-java, activemq, easymock, libapache-mod-jk, svnkit, sweethome3d-furniture-editor, sweethome3d-textures-editor, sweethome3d-furniture, sweethome3d, sweethome3d-furniture-nonfree, maven-enforcer, bsaf, libjcommon-java, libjfreechart-java, libcsv-java, jansi, jansi-native, jboss-xnio and undertow.
• The update of maven-enforcer caused a regression in its reverse-dependencies (#841197) and required a patch and upload of another revision. The Jfreechart library upgrade also required a compatibility patch for statcvs.
• I updated stegosuite and fixed an incorrect Section field in debian/control.
• I updated the Debian packaging of libjgraph-java and libjemmy2-java.
• I decided to work on getting Eclipse into shape again. The current version is outdated and affected by several bugs at the moment and I really think it should not be released with Stretch. The new version though requires a major package update because the build system is Maven based now. Luca Vercelli already worked on sat4j and Tycho, two preconditions for packaging Eclipse. I reviewed sat4j and uploaded an NMU (#815911) later. I m still working on Tycho (#816604) and I hope to finish it soon.
• Netbeans 8.2 was released. I have already completed the work on libnb-platform18-java (updated package is available in Git) but I haven t started yet with netbeans itself. I intend to continue the fun in November.

Debian LTS This was my eight month as a paid contributor and I have been paid to work 13 hours on Debian LTS, a project started by Rapha l Hertzog. In that time I did the following:

• From 10. October until 17. October I was in charge of our LTS frontdesk. I triaged bugs in libgd2, graphicsmagick, libxrender, mupdf, libxfixes, guile-2.0, glance, inspircd, libxi, libxv, libxst, spip, libxml2, libarchive and jasper.
• DLA-648-1. Issued a security update for c-ares fixing 1 CVE.
• DLA-664-1. Issued a security update for libxrender fixing 2 CVE.
• DLA-666-1. Issued a security update for guile-2.0 fixing 2 CVE.
• DLA-667-1. Issued a security update for libxv fixing 1 CVE.
• DLA-668-1. Issued a security update for libass fixing 2 CVE. I triaged CVE-2016-7970 and marked the version in Wheezy as not affected.
• DLA-673-1. Issued a security update for kdepimlibs fixing 1 CVE.

Non-maintainer uploads

• I fixed various RC bugs in gnudoq and xsok which are not maintained by the Games Team. The following games are available in Stretch again: gnudoq (#817296, #817484), xsok (#817738) and I also worked on four more bug fixes to improve the game s desktop integration and internationalization support.
• I fixed another RC bug in trackballs (#831119) but while I was working on the update I discovered that the game frequently segfaults which makes it kind of unplayable (#839788). I haven t found a solution yet but I suspect the switch to guile-2.0 and related patches introduced this behavior.

QA

• I uploaded a new revision of criticalmass and applied a patch from Adrian Bunk to fix #811816, a FTBFS.
• I triaged an RC bug for raptor2 (#824735) and the issue could be closed after the bug reporter confirmed that raptor2 built fine again.